BEAL is a deep active learning method that uses Bayesian deep learning with dropout to infer the model’s posterior predictive distribution and introduces an expected confidence-based acquisition function to select uncertain samples. Experiments show that BEAL outperforms other active learning methods, requiring fewer labeled samples for efficient training.
With deep learning, the ROI for having clean and high quality data is immense, and this is realized in every phase of training. For context, the era right before BERT in the text classification world was one where you wanted an abundance of data, even at the expense of quality. It was more important to have representation via examples than for the examples to be perfect. This is because many Al systems did not use pre-trained embeddings (or they weren't any good, anyway) that could be leveraged by a model to apply practical generalizability. In 2018, BERT was a breakthrough for down-stream text tasks,
